Safe Policies for Reinforcement Learning via Primal-Dual Methods

In this article, we study the design of controllers in context stochastic optimal control under assumption that model system is not available. This is, aim to a Markov decision process which do know transition probabilities, but have access sample trajectories through experience. We define safety as agent remaining desired safe set with high probability during operation time. The drawbacks formulation are twofold. problem nonconvex and computing gradients constraints respect policies prohibitive. Hence, propose an ergodic relaxation following advantages. 1) guarantees maintained case episodic tasks they hold until given time horizon for continuing tasks. 2) constrained optimization despite its nonconvexity has arbitrarily small duality gap if parametrization controller rich enough. 3) Lagrangian associated learning can be computed using standard reinforcement results approximation tools. Leveraging these advantages, exploit primal-dual algorithms find optimal. test proposed approach navigation task continuous domain. numerical show our algorithm capable dynamically adapting policy environment required levels.